create and validate build_variables.bzl #8326

swolchok · 2025-02-08T01:36:59Z

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

Differential Revision: D80187441

[ghstack-poisoned]

swolchok · 2025-02-08T01:37:00Z

Stack from ghstack (oldest at bottom):

pytorch-bot · 2025-02-08T01:37:03Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8326

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 7f2ca6a with merge base 70ab5ab ():

NEW FAILURE - The following job has failed:

Build documentation / build (buck2) / Build doc (gh)
At least one of the pre-conditions you specified did not hold

This comment was automatically generated by Dr. CI and updates every 15 minutes.

First step of #8268. ghstack-source-id: b29aa49 ghstack-comment-id: 2644414497 Pull Request resolved: #8326

[ghstack-poisoned]

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: 07b627b ghstack-comment-id: 2644414497 Pull Request resolved: #8326

swolchok · 2025-02-10T16:10:52Z

Current error:

Buck-generated _custom_ops__srcs does not match hardcoded CUSTOM_OPS_SRCS
    in build_variables.bzl.  Left:
    extension/llm/custom_ops/op_fallback.cpp;extension/llm/custom_ops/op_fast_hadamard_transform.cpp;extension/llm/custom_ops/op_sdpa.cpp;extension/llm/custom_ops/op_update_cache.cpp;extension/llm/custom_ops/spinquant/fast_hadamard_transform.cpp;extension/llm/custom_ops/spinquant/third-party/FFHT/fht_avx.c;kernels/portable/cpu/util/reduce_util.cpp


     Right: extension/llm/custom_ops/op_fallback.cpp;extension/llm/custom_ops/op_fast_hadamard_transform.cpp;extension/llm/custom_ops/op_sdpa.cpp;extension/llm/custom_ops/op_update_cache.cpp;extension/llm/custom_ops/spinquant/fast_hadamard_transform.cpp;extension/llm/custom_ops/spinquant/third-party/FFHT/fht_neon.c;kernels/portable/cpu/util/reduce_util.cpp

So, we need a mechanism for choosing fht_avx.c on Intel and fht_neon.c on ARM. Best way is probably to add a CMake-native way to do this and filter these files out of the Buck-to-CMake pipeline.

[ghstack-poisoned]

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: 91476c9 ghstack-comment-id: 2644414497 Pull Request resolved: #8326

[ghstack-poisoned]

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: b2ffb3b ghstack-comment-id: 2644414497 Pull Request resolved: #8326

[ghstack-poisoned]

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: dc50258 ghstack-comment-id: 2644414497 Pull Request resolved: #8326

[ghstack-poisoned]

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: c4195ce ghstack-comment-id: 2644414497 Pull Request resolved: #8326

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. ghstack-source-id: c4195ce ghstack-comment-id: 2644414497 Pull Request resolved: pytorch/executorch#8326

[ghstack-poisoned]

swolchok · 2025-08-13T17:51:39Z

@swolchok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

swolchok · 2025-08-13T20:11:42Z

@swolchok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Making the structure in this file mirror buck's worldview more closely when it makes sense. #8268 (Also piggybacked a fixup for a continue() I left for this target in #8326.)

First step of pytorch#8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation. Differential Revision: [D80187441](https://our.internmc.facebook.com/intern/diff/D80187441)

Making the structure in this file mirror buck's worldview more closely when it makes sense. pytorch#8268 (Also piggybacked a fixup for a continue() I left for this target in pytorch#8326.)

Update

07adf1a

[ghstack-poisoned]

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 8, 2025

swolchok added a commit that referenced this pull request Feb 8, 2025

WIP: create build_variables.bzl

1c67de8

First step of #8268. ghstack-source-id: b29aa49 ghstack-comment-id: 2644414497 Pull Request resolved: #8326

swolchok marked this pull request as draft February 8, 2025 01:37

Update

bc3fd39

[ghstack-poisoned]

swolchok added 3 commits February 10, 2025 11:00

Update

00ce927

[ghstack-poisoned]

Update

6440d3d

[ghstack-poisoned]

Update

e62191a

[ghstack-poisoned]

swolchok mentioned this pull request Feb 10, 2025

Add extension/llm/custom_ops/spinquant/test to CMake #8345

Merged

swolchok changed the base branch from main to gh/swolchok/243/head February 10, 2025 19:00

swolchok mentioned this pull request Feb 10, 2025

determine fht_avx.c/fht_neon.c based on CMake target processor #8346

Merged

swolchok added the release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc. label Feb 10, 2025

Update

630bdd0

[ghstack-poisoned]

swolchok changed the title ~~WIP: create build_variables.bzl~~ create and validate build_variables.bzl Feb 10, 2025

swolchok requested review from dbort, digantdesai and malfet February 10, 2025 19:26

swolchok marked this pull request as ready for review February 10, 2025 20:54

Update

098432f

[ghstack-poisoned]

swolchok added 3 commits February 11, 2025 16:28

Update

773ff37

[ghstack-poisoned]

Update

87513d8

[ghstack-poisoned]

Update

b61319b

[ghstack-poisoned]

swolchok mentioned this pull request Feb 12, 2025

build_variables.bzl: split PROGRAM_NO_PRIM_OPS_SRCS from EXECUTORCH_CORE_SRCS #8397

Merged

swolchok added 6 commits February 24, 2025 21:14

Update

25a26e2

[ghstack-poisoned]

Update

aafa130

[ghstack-poisoned]

Update

71ef8af

[ghstack-poisoned]

Update

b52f223

[ghstack-poisoned]

Update

8df2f1d

[ghstack-poisoned]

Update

7f5cfd3

[ghstack-poisoned]

Base automatically changed from gh/swolchok/243/head to main February 26, 2025 20:14

swolchok requested review from iseeyuan, jackzhxng and larryliu0820 as code owners February 26, 2025 20:14

swolchok added 2 commits August 12, 2025 16:12

rebase

aa296c7

[ghstack-poisoned]

Update

4184f07

[ghstack-poisoned]

swolchok added 2 commits August 12, 2025 20:35

Update

9dbb212

[ghstack-poisoned]

rebase atop revert to unbreak tests

7f2ca6a

[ghstack-poisoned]

GregoryComer approved these changes Aug 13, 2025

View reviewed changes

swolchok merged commit 52b45e2 into main Aug 13, 2025
104 of 106 checks passed

swolchok deleted the gh/swolchok/240/head branch August 13, 2025 20:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

create and validate build_variables.bzl #8326

create and validate build_variables.bzl #8326

Uh oh!

swolchok commented Feb 8, 2025 •

edited

Loading

Uh oh!

swolchok commented Feb 8, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Feb 8, 2025 •

edited

Loading

Uh oh!

swolchok commented Feb 10, 2025

Uh oh!

swolchok commented Aug 13, 2025

Uh oh!

swolchok commented Aug 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

create and validate build_variables.bzl #8326

create and validate build_variables.bzl #8326

Uh oh!

Conversation

swolchok commented Feb 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swolchok commented Feb 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8326

❌ 1 New Failure

Uh oh!

swolchok commented Feb 10, 2025

Uh oh!

swolchok commented Aug 13, 2025

Uh oh!

swolchok commented Aug 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

swolchok commented Feb 8, 2025 •

edited

Loading

swolchok commented Feb 8, 2025 •

edited

Loading

pytorch-bot bot commented Feb 8, 2025 •

edited

Loading